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Method and device for the registration of images 



The invention relates to a method for the registration of a series of at least 
three temporally successively acquired images of an object, and also to a corresponding 
registration device. The invention also relates to a device for the temporally successive 
acquisition of a series of medical image data from an examination zone of an object to be 
5 examined, as well as to a computer program for carrying out such a method and/or for 
controlling such a device. 

For the evaluation of temporally successively acquired images it is often 

I"* 

Q necessary to register such images during a first step of the image processing operation in 
f S order to compensate notably for undesirable motions that have occurred during the 
Wo acquisition of the images and that would falsify the evaluation result. Such motions may be 
in motions of the object itself as well as motions of the image acquisition device. Registration in 
' ^ this context is to be understood to mean that it is determined how the various images have to 
M be mapped on one another; notably the translations and/or rotations are determined whereto 
> J the object or the image acquisition device has been subject between the acquisition of the 
:.| 5 individual images that must be compensated prior to the evaluation of the images. 
i"^' For example, when temporally successively acquired images of the night sky 

have to be combined or compared so as to derive astronomic data, it is first necessary to 
eliminate any motions of the image acquisition device, for example of the stand of the 
camera, and to determine the exact positional relationship between the images. The 
20 evaluation result would otherwise be falsified. 

A further example in this respect is formed by medical image data, for 
example data from functional magnetic resonance tomography where often approximately 
100 three-dimensional images are acquired. Motions of the head of the patient during the 
image data acquisition would definitely degrade the evaluation result, that is, the resultant 
25 functional survey image. Compensation for such motions, therefore, is an absolute must. A 
high registration accuracy is also necessary since even slight inaccuracies and interpolation 
errors would already give rise to artifacts in the resultant functional survey images. 

According to known registration methods, for example as described in " 
Medical Image Matching - a review with classification", IEEE Eng. Med. Bio. 12, (1993) pp. 
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26-29, the registration is performed on the basis of image structures, for example boundary 
structures or special image characteristics, or on the basis of gray values of the images. In any 
case only two images are considered and a mapping rule is determined between only two 
images to be registered, said rule being used to transform one of the two images to the other so 
5 that they register. In order to determine the mapping rule, customarily use is made of a 
similarity measure, but also this measure is determined only for the two images that are 
examined and have to be registered. One of these two images represents the reference image 
with which the other image is registered. 

In the case of a series of temporally successively acquired images, all images are 
1 0 thus registered with a fixed reference image in the known methods, or each image is registered 
with the temporally preceding image that has already been registered. The available 
information, however, is not optunally used in that case, so that the registration accuracy is not 
Z. optimum. For example, it may occur that a reference image that has been selected for the 

registration is not optimum for all images to be registered, for example, because structures that 
15 are contained in many of the images to be registered are not shown in the reference image. In 
% the case of registration with the respective temporally preceding image, however, registration 
^ errors that have occurred during the first registration steps can be taken up in ftirther 
* registration steps and may accumulate or propagate through all registration steps. 

The invention, therefore, has for its object to provide a method and a device for 
^0 the registration of a series of temporally successively acquired images of an object in which 
said drawbacks have been mitigated and which notably yield an optimum registration result 
while making optimimi use of the information available in the images to be registered. 

This object is achieved by means of a method as disclosed in claim 1 and by 
means of a device as disclosed in claim 8. 
25 The invention is based on tiie recognition of the fact that for the registration of 

images, instead of observing each time only two images separately and determining the 
mapping rule on the basis of a similarity measure calculated for these two images only, it 
makes more sense to determine a similarity measure from the data of all images so as to 
determine the mapping rule for two images to be registered, or to determine mapping rules for 
30 more than two images, and to use said similarity measure so as to decide whether the quality of 
the current mapping rule used is adequate or not. 

The determination of the mapping rules is preferably performed iteratively as 
described in claim 2. This means that in a first step one or more mapping rules are selected for 
one or more images to be registered and that in a second step the similarity measure is 
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determined for all images to be registered, that is, not only for the image that is being treated at 
the relevant instant, the images examined during the first step bemg transformed by way of the 
selected mapping rules prior to the determination of the similarity measure. On the basis of the 
similarity measure determined in the second step it is then decided whether the mapping rules 
5 selected in the first step are optimum or require further optimization. In the first case one or 
more further images are examined and the steps 1 and 2 are performed again, whereas in the 
second case the procedure returns to the step 1 in order to execute a fiirther optimization of the 
mapping rules for the same images in an iterative manner. A limit value or an extreme value for 
the similarity measure can thus be used as a criterion for determining whether or not a mapping 
10 rule is optimimi. 

It is advantageous to examine each time only a single image to be registered and 
to optimize the corresponding mapping rule for this image in conformity with the described 
Q method. However, it is also feasible to vary the mapping rules for a plurality of images or all 

images simultaneously and to determine the similarity measure after each step of the variation 
-|5 process. 

Preferably, characteristic structures or characteristic image values in the images 
are used for the registration, it being possible to select different or identical characteristic 
features for each image to be registered. 

The mapping rules to be determined should preferably compensate for arbitrary 
20 geometrical changes of the object imaged, notably for translations, rotations, compressions and 
i expansions. However, it is also feasible to restrict the changes to be compensated, for example 
to translations. 

In a preferred embodiment of the invention the image registration concems a 
series of medical images of an object to be examined, notably a series of two-dimensional or 

25 three-dimensional images formed by means of a medical imaging modality. All of said images 
may have been acquired by means of the same medical imaging modality, for example a 
temporal series of images formed by means of X-ray fluoroscopy. However, the images may 
also have been acquired by means of different imaging modalities. Thus, it is also possible to 
register images that have been acquired by means of a magnetic resonance tomography 

3 0 apparatus with images that have been acquired by means of an ultrasound device. 

The invention can be used particularly advantageously for functional magnetic 
resonance tomography wherein temporal series of images of the brain are used for the study of 
brain activities in response to extemal stimuli. In order to arrive at reliable results for such a 
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study, a very high accuracy is required for the registration; such a high accuracy can be 
achieved by means of the invention. 

The invention also relates to a registration device as disclosed in claim 8, which 
device includes a storage unit for storing images and an arithmetic unit for determining the 
5 mapping rules and the similarity measure. The construction of this device may also be further 
elaborated so as to realize embodiments that are identical or similar to and compatible with the 
method in accordance with the invention as described above and also as disclosed iti the 
dependent claims for the method. 

The invention also relates to a device for the temporally successive acquisition 
10 of a series of medical image data from an examination zone of an object to be examined, 

which device includes a registration device in accordance with the invention. An acquisition 
device of this kind may be, for example a magnetic resonance tomography apparatus, a 

Is 

i.3 computed tomography apparatus, an X-ray device or an ultrasound device, 
r Finally, the invention also relates to a computer program for carrying out the 

15 method in accordance with the invention and/or for controlling the registration device in 
1 accordance with the invention. 

The invention will be described in detail hereinafter witii reference to the 
drawings. Therein: 

1 Fig. 1 is a block diagram of a first known version of a registration method, 

;|0 Fig. 2 is a block diagram of a second known version of a registration method, 

i Fig. 3 is a block diagram of a version of the registration method in accordance 

with the invention, and 

Fig. 4 is a block diagram of an image acquisition device in accordance with 

the invention. 

25 Fig. 1 shows a block diagram of a first version of a registration method in 

accordance with the state of the art. This Figure shows, by way of example, four images Ii, I2, 
I3, 14 of a series of n temporally successively acquired images li...In that are to be registered 
relative to one another. These images may be, for example temporally successively acquired 
two-dimensional or three-dimensional slice images of the brain of a patient that have been 

30 acquired by means of a magnetic resonance tomography apparatus in order to carry out 
functional studies of individual parts of the brain in response to external stimuli. For the 
registration it is necessary to derive mapping rules T2, T3, T4...Tn which are used to transform 
the original images I2, 13, l4...In into registered images Ii', I2', I3', Lt'-In' in order to ensure that 
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these images are registered with one another. The mapping rules T2, T3, T4...Tn thus serve to 
determine the positional relationships in space between the images Ii to I„. 

According to the version that is shown each time two neighboring images are 
registered, which means that first the image I2 is registered with the image Ii, resulting in the 
5 registered image I2'. Subsequently, the image I3 is registered with the previously registered 
image I2', resulting in the registered image I3'. Subsequently, the image I4 is registered with 
the registered image I3'; this yields the registered image I4'. 

Two images are registered with one another in such a manner that a first 
tentative mapping rule T2 is defined for the image I2, which rule is used to transform the 
1 0 image I2 into the image I2'. The similarity measure M2 is then determined between the image 
I2' obtained and the reference image Ii', in this case corresponding to the image Ii, which 
similarity measure forms an indication as to whether the similarity between the images I2' and 
r« ir (or Ii) is adequate so that the mapping rule T2 need not be further modified. The similarity 

measure may be determined as, for example the variances of voxels, that is, of individual 
15 small image points or groups of image points, in the images Ii' and I2'. When the similarity 
"J measure reveals that the similarity is not yet adequate, the mapping rule is step-wise varied 
until the similarity measure each time calculated indicates that the similarity between the 

s 

1^ images I2' and Ii' is good enough. 

*r The procedure for the registration of an image with the respective preceding, 

20 already registered image is identical to the procedure described above. This means that the 
,^ mapping rule T3 is determined via the similarity measure M3 between the images I3' and I2'. 
Analogously, the mapping rule T4 is then determined between the images I4' and I3' by way of 
the similarity measure M4. This procedure continues until all images have been registered 
with the respective preceding image. This method has the drawback that each image is 
25 always registered exclusively with the respective temporally preceding image, so that errors 
that occur in the first registration steps are propagated in flirther registration steps. 

Fig. 2 shows a second version of a registration method in conformity with the 
state of the art. In contrast with the version shown in Fig. 1, all images I2, 13, U--ln are now 
registered with a fixed reference image Ii' which again corresponds to the image Ii. The 
30 mapping rule T2 is then determined by means of the similarity measure M2 which is 

determined fi:om the images I2' and Ii' as in the version shown in Fig. 1 . Unlike in the version 
shown in Fig. 1 , the similarity measure M3 and the similarity measure M4 for determining 
the mapping rules T3 and T4 are determined firom the image I3' and the reference image Ii', 
and fi-om I4' and Ii', respectively. 
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This method, however, has the drawback that it may occur that not all 
structures that are present in the reference image Ii' and are used for the registration are also 
present in all other images, so that the registration takes place with different reference points. 
Conversely, it may also occur that structures that are present in the individual images h, h, 
U-.ln to be registered and are particularly suitable for the registration are absent in the 
reference image Ii'. 

Fig. 3 shows a block diagram of a version of the method in accordance with 
the invention. According to this version the mapping rules T2, T3, T4...Tn are all determined 
on the basis of the same similarity measure M which is determined from all images Ii', I2', I3', 
l4'...In'. The determination of the mapping rules T2, T3, T4...Tn can be performed in steps by 
determining first the mapping rule T2, after which the similarity measure M is determined 
jfrom all images h'-U' after each variation in order to check whether the mapping rule is good 
enough. This procedure can be continued until an optimxmi has been found for the mapping 
rule T2. Subsequently, in the next steps the mapping rule T3 can be determined and 
subsequently T4, that is, each time while utilizing the same similarity measure M firom all 
images. Alternatively, some or all mapping rules can be varied simultaneously and 
subsequently the similarity measure can be determined; this operation is then continued in an 
alternating fashion until an optimum has been found for the similarity measure M. 

Mathematically speaking, the similarity measure can be calculated, for 
example as a variance of a voxel that is based on a gray value (considered as a function of 
time), so as to be integrated over the entire image volume; to this end, use is made of the 
following formula: 

"Ijc^^^lz /=1 M i=l /=1 

Therein, Ii,...,In denote the images of a temporal series and Ix, ly and Iz denote 
the number of voxels along each co-ordinate axis. Moreover, it holds that: 
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which indicates the average gray value of a voxel. Moreover, a factor wyk can be introduced 
so as to make the effect of voxels that are situated at the edges of the image less than that of 
voxels that are situated at the at the center of an image. 

In order to optimize the similarity measure, n-1 mapping rules must be determined in 
the case of n images. When fixed transformations with three rotation parameters and three 
translation parameters are assumed, the total number of parameters thus amounts to 6n-6. 

The steps for the iterative determination of the mapping rules will be 
illustrated hereinafter on the basis of two versions. According to a first version the following 
steps are executed: 

1 . Registration of the first two images Ii (= Ii') and h yields the mapping rule T2. 
All other images l3,...,In are ignored during this step. The similarity measure is then given by: 

This yields I2'. 

2. When s images (Ii',...,Is') have been registered after a plurality of steps, the 
registration of the image Is+i and the determination of the mapping rule Tg+i are obtained by 
optimization of 

{^S + l)\.^lyL^ 1=1 y=l k=\ r=l 

for which it holds that: 
This results in I's+i- 



, PHDE000192 



8 09.10.2001 
3. This iterative procedure is terminated after n-1 iterations, after which Ii',...,In' 

are available. 

In conformity with a second version, all images Ii,...,In are simultaneously 

optimized. 

The method in accordance with the invention offers a significant improvement 
of the registration result, that is, of the mapping rules, in comparison with the registration 
result offered by the known methods. 

Fig. 4 shows a device for the temporally successive acquisition of a series of 
image data, that is, notably medical images of an examination zone of a patient. The device 1 
may be, for example a magnetic resonance tomography apparatus and includes a detector 2 
for the acquisition of the image data Ii, h, ... that is applied to the registration device 3. This 
device includes on the one hand a storage unit 4 for the storage of the image data and an 
arithmetic unit 5 for the calculation of the mapping rules and the similarity measure. The 
registered images Ii', I2', ... are applied from the device 3 to a further image evaluation unit 
that is not shown herein. 

The invention is restricted neither to the embodiments shown nor to the 
described application for medical image data. The invention can be used for the registration 
of arbitrary temporally successively acquired images of the same object for which a high 
registration accuracy is required in order to compensate notably for motions of the object or 
of the image acquisition device during the image acquisition. It is feasible, for example, to 
use the invention also for motion correction of digital video recordings. 

The method in accordance with the invention may be carried out by way of a 
suitable computer program that is executed by an image processing computer. Such a 
computer program may also be stored in the image data acquisition device and may control a 
corresponding registration device. 



